Add adapter crash-injection and mock cause (0070)#163
Merged
chris-colinsky merged 2 commits intoJun 17, 2026
Conversation
Implement proposal 0070's two conformance-adapter capabilities: a `crash_injection` directive (after_fan_out_instance / after_node) that simulates a checkpoint-boundary crash with no asserted first-run outcome, and a recursive mock `cause` that chains a flaky failure to an originating error. Wire conformance fixtures 067 (crash-injection fan-out resume) and 068 (failure-mock cause chain). A general per-instance execution recorder lets plain-node fan-outs report executed / skipped on resume, consulted as a fallback so the existing flaky_per_index fixtures are unchanged. The after_node boundary has a unit test (no fixture exercises it). Advance the pinned spec to v0.58.0; test-vocabulary only, no library behavior change.
There was a problem hiding this comment.
Pull request overview
Adds conformance-adapter vocabulary and harness support for spec v0.58.0 proposal 0070 (crash injection at checkpoint boundaries + recursive mock cause chaining), and updates the repo’s spec pin/version sync points accordingly.
Changes:
- Adds
crash_injectionhandling in the checkpoint conformance runner, including per-instance execution recording fallback for plain-node fan-outs. - Adds recursive
causesupport to failure mocks so fixtures can raise chained exceptions. - Bumps the pinned spec/version references from v0.57.0 → v0.58.0 across config, runtime constants, and docs.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_smoke.py | Updates spec version assertion to 0.58.0. |
| tests/conformance/test_pipeline_utilities.py | Extends failure-isolation fixture set to include fixture 068. |
| tests/conformance/test_checkpoint.py | Implements crash-injection parsing, abort swallowing for crash-only first runs, and execution-recording fallback; adds unit test for after_node. |
| tests/conformance/adapter.py | Adds recursive mock cause chaining; adds execution recorder wrapper and threads recorder map into graph builds. |
| src/openarmature/AGENTS.md | Updates embedded spec version text to 0.58.0. |
| src/openarmature/init.py | Bumps __spec_version__ to 0.58.0. |
| pyproject.toml | Bumps [tool.openarmature].spec_version to 0.58.0. |
| conformance.toml | Updates spec_pin to v0.58.0 and marks proposal 0070 implemented. |
| CHANGELOG.md | Updates spec pin summary to include v0.58.0 / proposal 0070. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address PR review on the crash_injection plumbing: - crash_injection now defines the crash boundary exclusively; when set, the legacy fan_out.abort_after_instance directive is ignored, so an instance-boundary and a node-boundary abort cannot both be active. - A non-dict crash_injection is coerced to None in the first-run handler, matching _find_crash_injection's mapping check, so a malformed directive no longer triggers the swallow path. - after_fan_out_instance now honors its documented node field: the parser threads the node name and _maybe_abort filters fan_out_progress by it, so a multi-fan-out graph aborts after the named node. Single-fan-out fixtures and the legacy path are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements proposal 0070's two new conformance-adapter capabilities and wires the two fixtures that exercise them. This is test-vocabulary only, with no library behavior change.
crash_injection(conformance-adapter §5.6): a top-level directive that simulates a crash at a checkpoint boundary independent of an instance failure, so resume can be tested from any saved state.after_fan_out_instance: {node, index}crashes after that instance's completion save;after_node: <node>crashes after the save recording that node. The first run has no asserted outcome (it "crashed"). It reuses the existing abort machinery, decoupled from thefirst_run_expected_errorassertion.cause(conformance-adapter §5.1): an optional recursivecause: {category, message, cause: {...}}on afailure_sequenceentry, so a flaky mock raises a chained exception. Fixture 068 uses it to pin proposal 0068's outermost-wins derivation (the two-non-carrier-link case fixture 066 left out).The crash/resume +
saved_record_assertions+instances_executed/skipped_during_resumevocabulary the proposal formalizes was already implemented. The one gap was per-instance execution tracking for a plain-node fan-out (fixture 067 uses a plainupdate_from_fieldscorer, not aflaky_per_indexnode). A general per-instance execution recorder fills that, consulted as a fallback only when noflaky_per_indexnode recorded, so existing retry-resume fixtures stay unchanged.Advances the pinned spec to v0.58.0. Second of four PRs folding the v0.14.0 consolidated spec-review findings (0068 through 0071) into the release.
Changes
tests/conformance/adapter.py:_build_mock_causewired into thefailure_sequenceraise;_wrap_with_execution_recorderand aninstance_execution_recordersparam (skipsflaky_per_indexnodes, which record their own).tests/conformance/test_checkpoint.py:crash_injectionparsing and the swallow path across the abort branches,_CapturingCheckpointer.abort_after_node, execution-recorder threading and fallback, fixture 067 wired in, and anafter_nodeunit test (no fixture exercises that boundary).tests/conformance/test_pipeline_utilities.py: fixture 068 wired in.conformance.toml, the spec-version sync points, regeneratedAGENTS.md;[proposals."0070"]marked implemented; changelog pin-summary updated.Testing
uv run pytest tests/: 1315 passed, 376 skipped.ruffandpyrightclean.